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DETAILED ACTION 

1 . This is in response to the applicant response filed on 04/29/2010. In the 
applicant's response, claim 1, 6-8, 12, 18, 34, 35, 37, 38, 47 and 61 were amended. 
Accordingly, claims 1-72 are pending and being examined. 

2. The non-statutory double patenting rejection is withdrawn in view of applicant's 
terminal disclaimer. 



Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



Claims 1-3, 10-12, 15, 17, 27-29, 32, 34-41, 44, 46-49, 52-54, 57, 59-60, 68-69 
and 71-72 are rejected under 35 U.S.C. 102(e), as being anticipated by Zon (US 
PUGPub 2003/01560973). 
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As to claim 1 , Zon discloses a user interaction system (the universal remote control 
system, see fig.3 and [0027] lines 1-7), comprising: 

an electrical apparatus (the monitor devices, see 200 in fig.3 and [0027] lines 6-7); 
a portable pointing device operable by a user for pointing to a region in space (the 
universal remote control system, see fig.3 and [0027] lines 1-7); 
a camera taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6-7); 

and 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing the picture (analyzing image, see [0033] lines 1-4), recognizing an 
object in the region (performing color matching/comparison for the video data, see 
[0033] lines 4-8), and transmitting user interface information derived from the picture to 
the electrical apparatus (using the communication channels and sending control 
signal to the devices, see 350, 360 in fig.3 and [0028] lines 1-8), wherein the camera 
is connected to the pointing device so that in operation it images the region pointed to 
(see [0027]). 

As to claims 2 and 68 , Zon further discloses wherein the user interface information 
includes apparatus control data for controlling operation of the electrical apparatus 
(permitting signals and data generated by the camera to be applied to control the 
monitors, see [0028] lines 1-8). 
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As to claims 3, 69 , Zon further discloses wherein the digital signal processor has an 
object characterizing means for characterizing an object or part of the object present in 
the picture of the region imaged by the camera, by providing first object characterizing 
features to object identification means for identifying the object (analyzing/comparing 
the data generated by one of the monitors to determine a monitor specific set of 
correction commands, see [0033] lines 8-11), and which object identification means 
is capable of outputting object identification data from which the user interface 
information is constructed (generating index data in a LUT for retrieving correction 
commands, see [0033] lines 14-24). 

As to claims 10, 27, 39, 52 , Zon further disclose, comprising a camera and being 
capable of sending a picture to a digital signal processor (the video camera, see 100 
in fig 3 and [0027] lines 6-7). 

As to claims 1 1 , 28, 40, 53 , Zon further disclose, wherein the pointing device is capable 
of sending a picture to the digital signal processor (sending image from camera 100 
to processor 330, see fig.3), which is capable of sending user interface information to 
an electrical apparatus based on the picture (using the communication channels and 
sending control signal to the devices, see 350, 360 in fig.3 and [0028] lines 1-8). 
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As to claims 12, 29, 41 and 54 , Zon further discloses wherein the digital signal 
processor (120) is comprised in the pointing device (the image analysis unit, see 334 
in fig.3 and [0029] lines 11-14 and [0033] lines 8-11). 

As to claims 1 5, 32, 44, 57 and 71 , Zon further discloses, comprising a programmable 
user interface code generator and a wireless transmitter for transmitting the code to the 
electrical apparatus (the programmable ROM, see [0029] lines 14-18). 

As to claims 17, 34, 46, 59 and 72 , Zon discloses the Electrical apparatus for use in a 
user interaction system as claimed in claim 1, comprising means which allow the 
electrical apparatus to send information about supported commands to a pointing 
device, based on an a call of the pointing device to the electrical apparatus (display 
menu command, see [0031] lines 13-17). 

As to claim 35 , Zon discloses a User interaction system (the universal remote control 
system, see fig.3 and [0027] lines 1-7), comprising: a first electrical apparatus (the 
monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a portable pointing device 
operable by a user for pointing to a region in space (the universal remote control 
system, see fig.3 and [0027] lines 1-7); 

a camera connected to the pointing device so that in operation it images the region 
pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); 
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a digital signal processor (the data analyzer, see 334 in fig.3), ), recognizing an object 
in the region (performing color matching/comparison for the video data, see [0033] 
lines 4-8), and receiving and processing data of said picture, and capable of 
transmitting user interface information derived on the basis of said picture data to the 
first electrical apparatus (using the communication channels and sending control 
signal to the devices, see 350, 360 in fig.3 and [0028] lines 1-8); and 
an object displaying a characteristic pattern to facilitate recognition of said object by the 
digital signal processor (the monitor 200b, see 200a in fig.3 and [0027] lines 6-7). 

As to claim 36 , Zon discloses the User interaction system as claimed in claim 35, 
wherein the object displaying a characteristic pattern is a second electrical apparatus 
(the monitor 200b, see 200a in fig.3 and [0027] lines 6-7). 

As to claims 37 , Zon discloses the User interaction system as claimed in claim 36, 
wherein the second electrical apparatus comprises a display (the monitor 200b, see 
200a in fig.3 and [0027] lines 6-7). 

As to claims 38 , Zon discloses the User interaction system as claimed in claim 35, 
wherein the object displaying a characteristic pattern is the first electrical apparatus (the 
monitor 200a, see 200a in fig.3 and [0027] lines 6-7). 
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As to claim 47 , Zon discloses a User interaction system (the universal remote control 
system, see fig.3 and [0027] lines 1-7), comprising: an electrical apparatus (the 
monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a portable pointing device 
operable by a user for pointing to a region in space (the universal remote control 
system, see fig.3 and [0027] lines 1-7); 

a camera connected to the pointing device so that in operation it images the region 
pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing data of said picture, recognizing an object in the region (performing 
color matching/comparison for the video data, see [0033] lines 4-8), and 
transmitting user interface information derived on the basis of said picture data to the 
electrical apparatus (using the communication channels and sending control 
signal to the devices, see 350, 360 in fig.3 and [0028] lines 1-8); and 
wherein said user interface information includes a specification of the electrical 
apparatus, or of a part of the electrical apparatus intended to be used by the user (the 
display menu commands which is corresponding to the aimed device, see [0031] 
lines 13-17). 

As to claim 48 . Zon discloses the User interaction system as claimed in claim 47, 
wherein the electrical apparatus, or the intended part of the electrical apparatus is 
identified by determining the position of the electrical apparatus in the picture of the 
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region pointed to (the camera located to take the entire image of the device, see 
[0032] lines 5-8). 

As to claim 49 , Zon discloses the User interaction system as claimed in claim 48, 
wherein the electrical apparatus, or the intended part of the electrical apparatus 
corresponds to a fixed position in the picture (the camera located to take the entire 
image of the device, see [0032] lines 5-8). 

As to claim 60 , Zon discloses a User interaction system (the universal remote control 
system, see fig.3 and [0027] lines 1-7), comprising: 

an electrical apparatus (the monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a 
portable pointing device operable by a user for pointing to a region in space; a camera 
taking a picture (the universal remote control system, see fig.3 and [0027] lines 1- 

7); and 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing the picture, and capable of transmitting user interface information 
derived from the picture to the electrical apparatus (using the communication 
channels and sending control signal to the devices, see 350, 360 in fig.3 and 
[0028] lines 1-8), wherein the camera is connected to the pointing device so that in 
operation it images the region pointed to characterized in that the system further 
comprises at least one localization beacon that can emit electromagnetic radiation, 
which can be captured by the system and for use to the digital signal processor in order 
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to recognize to where the pointing device is pointing by using information derived from 
the captured electromagnetic radiation (analog/digital signal generated by the video 
camera, see [0032] lines 5-10). 



Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Claims 4, 9,13-14,16,1 8-26, 30-31 , 33, 42-43, 45, 50-51 , 55-56, 58, 61-67 and 
70 are rejected under 35 U.S.C. 103(a), as being unpatentable over Zon in view of 
Silverstein (EP 1130906). Silverstein is cited by applicant in IDS and filed on 
12/14/2005. 

As to claims 4, 19, 23, 65 , Zon does not disclose motion trajectory estimation means for 
estimating a motion trajectory of the pointing device. However, Silverstein does teach a 
digital signal camera to be used as a pointing device, comprising: 
motion trajectory estimation means for estimating a motion trajectory of the pointing 
device and outputting a first motion characterizing signature, a signature being a 
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mathematical abstraction of the motion trajectory (the motion detection unit, see 130 
in fig.1 and [0013] lines 4-6); and 

signature identification means for identifying the first motion characterizing signature 
and outputting command identification data, which represents a user interaction 
command, corresponding with the first motion characterizing signature, from which 
command identification data the user interface information is constructed (the target 
icon selection unit which is corresponding to the movement of the point device, 
see [0016] lines 16-22). 

It would have been obvious to person skilled in the art at the time of the invention to 
combine the teaching of Silverstein into the teaching of Zon in order to robustly and 
accurately permitting cursor control and designation in digital camera display 
(Silverstein, [0006]). 

As to claims 9, 20, 63 , the combination of Zon and Silverstein further discloses the first 
motion characterizing signature is derived on the basis of successive pictures imaged 
by the camera at respective instances of time (Silverstein, using the changes 
between subsequent images to calculate a mount and direction of the motion, see 
[0012] lines 39-41). 

As to claims 13. 21 . 30, 42, 55, 62 , Zon discloses the Pointing device as claimed in 
claim 10. Zon does not disclose comprising motion sensing means for sensing a motion 
trajectory of the pointing device. However, Silverstein does teach comprising motion 
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sensing means for sensing a motion trajectory of the pointing device (motion sensor 
such as gyroscopes, see [0011] lines 17-19, and [0012] lines 41-50). It would have 
been obvious to person skilled in the art at the time of the invention to combine the 
teaching of Silverstein into the teaching of Zon in order to robustly and accurately 
permitting cursor control and designation in digital camera display (Silverstein, [0006]). 

As to claims 14, 31 , 43, 56 and 70, Zon discloses the Pointing device as claimed in 
claim 10. Zon does not disclose comprising a characteristic projector for optically 
projecting a characteristic pattern towards a region pointed to. However, Silverstein 
does teach comprising a characteristic projector for optically projecting a characteristic 
pattern towards a region pointed to (viewfinder, see 160 in fig.1 and [0017]). It would 
have been obvious to person skilled in the art at the time of the invention to combine the 
teaching of Silverstein into the teaching of Zon in order to robustly and accurately 
permitting cursor control and designation in remote control device (Silverstein, [0006]). 

As to claims 16. 25-26. 33. 45. 50-51. 58. 66-67 . Zon discloses the Pointing device as 
claimed in claim 10. Zon does not disclose comprising feedback means for feedback of 
user interface information. However, Silverstein does teach comprising feedback 
means for feedback of user interface information (displaying view field selected by 
the viewfinder, see [0032] lines 24-27). It would have been obvious to person skilled 
in the art at the time of the invention to combine the teaching of Silverstein into the 
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teaching of Zon in order to robustly and accurately permitting cursor control and 
designation in digital camera display (Silverstein, [0006]). 

As to claim 18 , Zon discloses a User interaction system, comprising: an electrical 
apparatus (the monitor devices, see 200 in fig.3 and [0027] lines 6-7); a portable 
pointing device operable by a user for pointing to a region in space (the universal 
remote control system, see fig.3 and [0027] lines 1-7); 
a camera connected to the pointing device so that in operation it images the region 
pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); and a digital signal processor (the data analyzer, see 334 in fig.3), capable of 
receiving and processing data of said picture, , recognizing an object in the region 
(performing color matching/comparison for the video data, see [0033] lines 4-8), 
and transmitting user interface information derived on the basis of said picture data to 
the electrical apparatus (using the communication channels and sending control 
signal to the devices, see 350, 360 in fig.3, [0028] lines 1-8, and [0027]). Zon does 
not disclose motion sensing means for estimating the motion of the pointing device. 
However, Silverstein does teach motion sensing means for estimating the motion of the 
pointing device (the motion detection unit, see 130 in fig.1 and [0013] lines 4-6). It 
would have been obvious to person skilled in the art at the time of the invention to 
combine the teaching of Silverstein into the teaching of Zon in order to robustly and 
accurately permitting cursor control and designation in digital camera display 
(Silverstein, [0006]). 
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As to claims 22, 61 and 64 , the combination of Zon and Silverstein discloses the User 
interaction system as claimed in claim 18 wherein the transmitted user interface 
information includes at least one feature selected from the group consisting of motion 
speed, motion direction, and acceleration of the pointing device (Silverstein, 
calculating direction of motion, see [0012] lines 39-41). 

As to claim 24 , the combination of Zon and Silverstein discloses the User interaction 
system as claimed in claim 18, further comprising room localization beacons for emitting 
electromagnetic radiation, wherein the digital signal processor is arranged to recognize 
to which part of the room the pointing device is pointing (analog/digital signal 
generated by the video camera, see [0032] lines 5-10). 

Allowable Subject Matter 

5. Claims 5-8 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

6. The following is a statement of reasons for the indication of allowable subject 
matter: 
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As to claims 5-6, no prior art in record teaches: "wherein the digital signal processor 
includes identification improvement means, which are capable of further improving a 
probability that the object represented as object identification data, and user interaction 
command represented as command identification data, are more reliably identified 
based on predetermined rules, yielding more reliable user interface information". 

As to claim 7 , no prior art in record teaches: "wherein the digital signal processor 
includes object association means for providing to the object identification means object 
association data comprising at least one of the data entities being: associated object 
characterizing features and object related data, the object association data being 
derivable from object template data in object memory originating from at least one of the 
methods: 

the object template data is obtained from object training means performing a 
predetermined calculation on second object characterizing features outputted by object 
characterizing means; and the object template data is derived from inputted object 
data." 

As to claim 8 , no prior art in record teaches: "wherein the digital signal processor 
includes signature association means for providing to the signature identification means 
signature association data comprising at least one of the data entities being: associated 
signature features and command related data, the signature association data being 
derivable from signature template data in signature memory originating from at least 
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one of the methods: 

the signature template data is obtained from signature training means performing a 
predetermined calculation on a second motion characterizing signature outputted by the 
motion trajectory estimating means ; and 

the command template data is derived from inputted command data." 

Response to Arguments 

7. Applicant's arguments with respect to claims 1-72 have been considered but are 
moot in view of the new ground(s) of rejection. 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. 
The examiner can normally be reached on 8:30am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571 -272-741 3. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/RUIPING LI/ 
Examiner, Art Unit 2624 
5/31 //2010 
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Supervisory Patent Examiner, Art Unit 2624 



